{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7edf5ae5",
   "metadata": {},
   "source": [
    "## Regexes in Python and Their Uses\n",
    "\n",
    "In this tutorial, you’ll explore regular expressions, also known as regexes, in Python. A regex is a special sequence of characters that defines a pattern for complex string-matching functionality.\n",
    "\n",
    "Imagine you have a string object s. Now suppose you need to write Python code to find out whether s contains the substring '123'. There are at least a couple ways to do this. You could use the in operator:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27dd1122",
   "metadata": {},
   "outputs": [],
   "source": [
    "s = 'foo123bar'\n",
    "'1234' in s"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3aa622e",
   "metadata": {},
   "source": [
    "If you want to know not only whether '123' exists in s but also where it exists, then you can use __.find() or .index()__. Each of these returns the character position within s where the substring resides:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "732bea9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "s = 'foo123bar'\n",
    "s.find('1234')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "096f67ad",
   "metadata": {},
   "source": [
    "In these examples, the matching is done by a straightforward character-by-character comparison. That will get the job done in many cases. But sometimes, the problem is more complicated than that.\n",
    "\n",
    "For example, rather than searching for a fixed substring like '123', suppose you wanted to determine whether a string contains any three consecutive decimal digit characters, as in the strings 'foo123bar', 'foo456bar', '234baz', and 'qux678'.\n",
    "\n",
    "Strict character comparisons won’t cut it here. This is where regexes in Python come to the rescue."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f202a93",
   "metadata": {},
   "source": [
    "### A (Very Brief) History of Regular Expressions\n",
    "In 1951, mathematician __Stephen Cole Kleene__ described the concept of a regular language, a language that is recognizable by a finite automaton and formally expressible using regular expressions. In the mid-1960s, computer science pioneer __Ken Thompson__, one of the original designers of Unix, implemented pattern matching in the QED text editor using Kleene’s notation.\n",
    "\n",
    "Since then, regexes have appeared in many programming languages, editors, and other tools as a means of determining whether a string matches a specified pattern. Python, Java, and Perl all support regex functionality, as do most Unix tools and many text editors.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "396ba522",
   "metadata": {},
   "source": [
    "###  The re Module\n",
    "Regex functionality in Python resides in a module named __re__. The re module contains many useful functions and methods, most of which you’ll learn about in the next tutorial in this series.\n",
    "\n",
    "For now, you’ll focus predominantly on one function, __re.search().__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7924c69",
   "metadata": {},
   "source": [
    "__re.search(regex, string)__\n",
    "\n",
    "Scans a string for a regex match."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57bc3c20",
   "metadata": {},
   "source": [
    "re.search(regex, string) scans string looking for the first location where the pattern regex matches. If a match is found, then re.search() returns a match object. Otherwise, it returns None."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e3db32f",
   "metadata": {},
   "source": [
    "### How to Import re.search()\n",
    "Because search() resides in the re module, you need to import it before you can use it. One way to do this is to import the entire module and then use the module name as a prefix when calling the function:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "369b86e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "from re import search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7597fab1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='123'>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s= 'foo123abc'\n",
    "re.search('123', s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "58450b17",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(6, 9), match='abc'>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "search('abc',s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "165a6901",
   "metadata": {},
   "source": [
    "### setup a pattern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ce7f441",
   "metadata": {},
   "outputs": [],
   "source": [
    "pattern = re.compile(r'123')\n",
    "match = re.search(pattern,s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "18573001",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\tTab\n"
     ]
    }
   ],
   "source": [
    "print('\\tTab')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4cc764a5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\\tTab\n"
     ]
    }
   ],
   "source": [
    "print(r'\\tTab')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca1b4c09",
   "metadata": {},
   "source": [
    "For the moment, the important point is that __re.search()__ did in fact return a match object rather than None. That tells you that it found a match. In other words, the specified __'regex'__ pattern 123 is present in s."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9357775",
   "metadata": {},
   "source": [
    "A match object is truthy, so you can use it in a __Boolean context__ like a conditional statement:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "699fbe17",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found a match.\n"
     ]
    }
   ],
   "source": [
    "if re.search('123', s):\n",
    "...     print('Found a match.')\n",
    "... else:\n",
    "...     print('No match.')\n",
    "..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a22f210",
   "metadata": {},
   "source": [
    "* The interpreter displays the match object as <_sre.SRE_Match object; span=(3, 6), match='123'>. This contains some useful information.\n",
    "\n",
    "* span=(3, 6) indicates the portion of <string> in which the match was found. This means the same thing as it would in slice notation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9feed3fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "s[3:6]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93384cbf",
   "metadata": {},
   "source": [
    "* In this example, the match starts at character position 3 and extends up to but not including position 6.\n",
    "\n",
    "* __match='123'__ indicates which characters from <string> matched."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf58d06e",
   "metadata": {},
   "source": [
    "This is a good start. But in this case, the <regex> pattern is just the plain string '123'. The pattern matching here is still just character-by-character comparison, pretty much the same as the in operator and .find() examples shown earlier. The match object helpfully tells you that the matching characters were '123', but that’s not much of a revelation since those were exactly the characters you searched for."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd5508ee",
   "metadata": {},
   "source": [
    "### Python Regex Metacharacters\n",
    "\n",
    "* The real power of regex matching in Python emerges when regex contains special characters called metacharacters. These have a unique meaning to the regex matching engine and vastly enhance the capability of the search.\n",
    "\n",
    "* Consider again the problem of how to determine whether a string contains any three consecutive decimal digit characters.\n",
    "\n",
    "* In a regex, a set of characters specified in square brackets ([]) makes up a character class. This metacharacter sequence matches any single character that is in the class, as demonstrated in the following example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6ea5eec4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='123'>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = 'foo123bar'\n",
    "re.search('[0-9][0-9][0-9]', s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab532a0f",
   "metadata": {},
   "source": [
    "* [0-9] matches any single decimal digit character—any character between '0' and '9', inclusive. \n",
    "* The full expression [0-9][0-9][0-9] matches any sequence of three decimal digit characters. In this case, s matches because it contains three consecutive decimal digit characters, '123'.\n",
    "\n",
    "These strings also match: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4a31d0d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='456'>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[0-9][0-9][0-9]', 'foo456bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "dea2877b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='234'>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[0-9][0-9][0-9]', '234baz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1e15a535",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='678'>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[0-9][0-9][0-9]', 'qux678')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f100215",
   "metadata": {},
   "source": [
    "On the other hand, a string that doesn’t contain three consecutive digits won’t match:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a42beb67",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('[0-9][0-9][0-9]', '12foo34'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35b0a466",
   "metadata": {},
   "source": [
    "* With regexes in Python, you can identify patterns in a string that you wouldn’t be able to find with the in operator or with string methods.\n",
    "\n",
    "* Take a look at another regex metacharacter. __The dot (.) metacharacter matches any character except a newline, so it functions like a wildcard:__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5509b0ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "s = 'foo123bar'\n",
    "re.search('1.3', s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a4a52e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "s = 'foo13bar'\n",
    "print(re.search('1.3', s))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59096e9c",
   "metadata": {},
   "source": [
    "* In the first example, the regex 1.3 matches '123' because the '1' and '3' match literally, and the . matches the '2'. Here, you’re essentially asking, “Does s contain a '1', then any character (except a newline), then a '3'?” The answer is yes for 'foo123bar' but no for 'foo13bar'.\n",
    "\n",
    "* These examples provide a quick illustration of the power of regex metacharacters. Character class and dot are but two of the metacharacters supported by the re module. There are many more. Next, you’ll explore them fully."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f5a64d3",
   "metadata": {},
   "source": [
    "## Metacharacters Supported by the re Module\n",
    "\n",
    "The following table briefly summarizes all the metacharacters supported by the re module. Some characters serve more than one purpose:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98d2f490",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "00ea41ab",
   "metadata": {},
   "source": [
    "The metacharacter sequences in this section try to match a single character from the search string. When the regex parser encounters one of these metacharacter sequences, a match happens if the character at the current parsing position fits the description that the sequence describes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22cb7fb8",
   "metadata": {},
   "source": [
    "### __1. []: Specifies a specific set of characters to match.__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "825bb749",
   "metadata": {},
   "source": [
    "* Characters contained in square brackets ([]) represent a character class—an enumerated set of characters to match from. A character class metacharacter sequence will match any single character contained in the class.\n",
    "\n",
    "You can enumerate the characters individually like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b2c6babb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='bar'>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('ba[artz]', 'foobarqux')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "6a20cea3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='baz'>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('ba[artz]', 'foobazqux')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c0dd3e8",
   "metadata": {},
   "source": [
    "The metacharacter sequence [artz] matches any single 'a', 'r', 't', or 'z' character. In the example, the regex ba[artz] matches both 'bar' and 'baz' (and would also match 'baa' and 'bat')."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78c475f7",
   "metadata": {},
   "source": [
    "* A character class can also contain a range of characters separated by a hyphen (-), in which case it matches any single character within the range. For example, [a-z] matches any lowercase alphabetic character between 'a' and 'z', inclusive:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "bebd6383",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='b'>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[a-z]', 'FOObar')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a98f7d36",
   "metadata": {},
   "source": [
    "[0-9] matches any digit character:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "6a63519e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 5), match='12'>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[0-9][0-9]', 'foo123bar')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b2dd550",
   "metadata": {},
   "source": [
    "* In this case, [0-9][0-9] matches a sequence of two digits. The first portion of the string 'foo123bar' that matches is '12'."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec47aa97",
   "metadata": {},
   "source": [
    "* [0-9a-fA-F] matches any hexadecimal digit character:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "aa48828e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(4, 5), match='a'>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[0-9a-fA-F]', '--- a0 ---')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0d64b41",
   "metadata": {},
   "source": [
    "Here, [0-9a-fA-F] matches the first hexadecimal digit character in the search string, 'a'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42c49c9c",
   "metadata": {},
   "source": [
    "__Note:__ In the above examples, the return value is always the leftmost possible match. re.search() scans the search string from left to right, and as soon as it locates a match for <regex>, it stops scanning and returns the match."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "515b3e2c",
   "metadata": {},
   "source": [
    "* You can complement a character class by specifying ^ as the first character, in which case it matches any character that isn’t in the set. In the following example, [^0-9] matches any character that isn’t a digit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "779c1e90",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('[^0-9]', '12345foo')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0485ae16",
   "metadata": {},
   "source": [
    "* If a ^ character appears in a character class but isn’t the first character, then it has no special meaning and matches a literal '^' character:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "2b97da9a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='^'>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[#:^]', 'foo^bar:baz#qux')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8321ea31",
   "metadata": {},
   "source": [
    "* As you’ve seen, you can specify a range of characters in a character class by separating characters with a hyphen. What if you want the character class to include a literal hyphen character? You can place it as the __first__ or __last__ character or escape it with a __backslash (\\):__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "69e4f5cc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='-'>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[-abc]', '123-456')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1c15558a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='-'>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[abc-]', '123-456')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f17eb71b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='-'>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[ab\\-c]', '123-456')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1632ffa",
   "metadata": {},
   "source": [
    "* If you want to include a literal ']' in a character class, then you can place it as the first character or escape it with backslash:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "930f3cd4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(5, 6), match=']'>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[]]', 'foo[1]')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "9e3bdca2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(5, 6), match=']'>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[ab\\]cd]', 'foo[1]')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8cea3e4",
   "metadata": {},
   "source": [
    "* Other regex metacharacters lose their special meaning inside a character class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "5742e66c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='*'>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[)*+|]', '123*456')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2372d524",
   "metadata": {},
   "source": [
    "* As you saw in the table above, * and + have special meanings in a regex in Python. They designate repetition, which you’ll learn more about shortly. But in this example, they’re inside a character class, so they match themselves literally.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dbe4a16",
   "metadata": {},
   "source": [
    "### 2.__dot (.):__ Specifies a wildcard"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2db2b2f5",
   "metadata": {},
   "source": [
    "The . metacharacter matches any single character except a newline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "3c9ca04c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 7), match='fooxbar'>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('foo.bar', 'fooxbar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac8af846",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('foo.bar', 'foobar'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb076c32",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('foo.bar', 'foo\\nbar'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b387bfc8",
   "metadata": {},
   "source": [
    "* As a regex, foo.bar essentially means the characters 'foo', then any character except newline, then the characters 'bar'. The first string shown above, 'fooxbar', fits the bill because the . metacharacter matches the 'x'.\n",
    "\n",
    "* The second and third strings fail to match. In the last case, although there’s a character between 'foo' and 'bar', it’s a newline, and by default, the . metacharacter doesn’t match a newline. There is, however, a way to force . to match a newline, which you’ll learn about at the end of this tutorial.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb0e6bdd",
   "metadata": {},
   "source": [
    "### __3. \\w, \\W: Match based on whether a character is a word character.__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56ccaf12",
   "metadata": {},
   "source": [
    "* \\w matches any alphanumeric word character. Word characters are uppercase and lowercase letters, digits, and the underscore (_) character, so \\w is essentially shorthand for [a-zA-Z0-9_]:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "ad55f17f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='a'>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('\\w', '#(.a$@&')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "b45b5102",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='a'>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('[a-zA-Z0-9_]', '#(.a$@&')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a63155ed",
   "metadata": {},
   "source": [
    "In this case, the first word character in the string '#(.a$@&' is 'a'."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1496b0f4",
   "metadata": {},
   "source": [
    "* \\W is the opposite. It matches any non-word character and is equivalent to [^a-zA-Z0-9_]:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d6cea3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('\\W', 'a_1*3Qb')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b90b124",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('[^a-zA-Z0-9_]', 'a_1*3Qb')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9df5a12f",
   "metadata": {},
   "source": [
    "Here, the first non-word character in 'a_1*3!b' is '*'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd90bb33",
   "metadata": {},
   "source": [
    "### 4. __\\d, \\D: Match based on whether a character is a decimal digit__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f69cba0",
   "metadata": {},
   "source": [
    "* \\d matches any decimal digit character. \n",
    "* \\D is the opposite. It matches any character that isn’t a decimal digit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "290f0ebc",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('\\d', 'abc4def')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06cd2603",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('\\D', '234Q678')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12b53a1c",
   "metadata": {},
   "source": [
    "* \\d is essentially equivalent to [0-9], and \\D is equivalent to [^0-9]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daaafd58",
   "metadata": {},
   "source": [
    "### 4. \\s, \\S: Match based on whether a character represents whitespace."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12684d86",
   "metadata": {},
   "source": [
    "* \\s matches any whitespace character: [\\ \\t\\n\\r\\f\\v]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c3049c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('\\s', 'foo\\nbar baz')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "148f5242",
   "metadata": {},
   "source": [
    "Note that, unlike the dot wildcard metacharacter, \\s does match a newline character."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "863885a4",
   "metadata": {},
   "source": [
    "\\S is the opposite of \\s. It matches any character that isn’t whitespace:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e1f3ce6",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('\\S', '  \\n foo  \\n  ')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d83da0d",
   "metadata": {},
   "source": [
    "* Again, \\s and \\S consider a newline to be whitespace. In the example above, the first non-whitespace character is 'f'.\n",
    "\n",
    "* The character class sequences \\w, \\W, \\d, \\D, \\s, and \\S can appear inside a square bracket character class as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9417f7f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('[\\d\\w\\s]', '---3---')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf08ec25",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('[\\d\\w\\s]', '---a---')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36689cb9",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('[\\d\\w\\s]', '--- ---')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e31c567",
   "metadata": {},
   "source": [
    "In this case, [\\d\\w\\s] matches any digit, word, or whitespace character. And since \\w includes \\d, the same character class could also be expressed slightly shorter as [\\w\\s]."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffac0995",
   "metadata": {},
   "source": [
    "### Escaping Metacharacters\n",
    "\n",
    "Occasionally, you’ll want to include a metacharacter in your regex, except you won’t want it to carry its special meaning. Instead, you’ll want it to represent itself as a literal character."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "613d85fb",
   "metadata": {},
   "source": [
    "#### 1. backslash (\\\\): Removes the special meaning of a metacharacter."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1befe193",
   "metadata": {},
   "source": [
    "* As you’ve just seen, the backslash character can introduce special character classes like word, digit, and whitespace. There are also special metacharacter sequences called anchors that begin with a backslash, which you’ll learn about below.\n",
    "\n",
    "* When it’s not serving either of these purposes, the backslash escapes metacharacters. A metacharacter preceded by a backslash loses its special meaning and matches the literal character instead. Consider the following examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3fc9570",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('.', 'foo.bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "77c4d8f9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='.'>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('\\.', 'foo.bar')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "775c1d17",
   "metadata": {},
   "source": [
    "* In the regex on line 1, the dot (.) functions as a wildcard metacharacter, which matches the first character in the string ('f'). \n",
    "* The . character in the regex on line 4 is escaped by a backslash, so it isn’t a wildcard. It’s interpreted literally and matches the '.' at index 3 of the search string.\n",
    "\n",
    "* Using backslashes for escaping can get messy. Suppose you have a string that contains a single backslash:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "f6d01a1f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "foo\\bar\n"
     ]
    }
   ],
   "source": [
    "s = r'foo\\bar'\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "904d5b00",
   "metadata": {},
   "source": [
    "Now suppose you want to create a regex that will match the backslash between 'foo' and 'bar'. The backslash is itself a special character in a regex, so to specify a literal backslash, you need to escape it with another backslash. If that’s that case, then the following should work:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "a30b8dcf",
   "metadata": {},
   "outputs": [
    {
     "ename": "error",
     "evalue": "bad escape (end of pattern) at position 0",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31merror\u001b[0m                                     Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-28-9674131d1684>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mre\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msearch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'\\\\'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/opt/anaconda3/lib/python3.8/re.py\u001b[0m in \u001b[0;36msearch\u001b[0;34m(pattern, string, flags)\u001b[0m\n\u001b[1;32m    199\u001b[0m     \"\"\"Scan through string looking for a match to the pattern, returning\n\u001b[1;32m    200\u001b[0m     a Match object, or None if no match was found.\"\"\"\n\u001b[0;32m--> 201\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0m_compile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpattern\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mflags\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msearch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstring\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    202\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    203\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0msub\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpattern\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrepl\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstring\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcount\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mflags\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/anaconda3/lib/python3.8/re.py\u001b[0m in \u001b[0;36m_compile\u001b[0;34m(pattern, flags)\u001b[0m\n\u001b[1;32m    302\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0msre_compile\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0misstring\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpattern\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    303\u001b[0m         \u001b[0;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"first argument must be string or compiled pattern\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 304\u001b[0;31m     \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msre_compile\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompile\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpattern\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mflags\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    305\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mflags\u001b[0m \u001b[0;34m&\u001b[0m \u001b[0mDEBUG\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    306\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_cache\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>=\u001b[0m \u001b[0m_MAXCACHE\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/anaconda3/lib/python3.8/sre_compile.py\u001b[0m in \u001b[0;36mcompile\u001b[0;34m(p, flags)\u001b[0m\n\u001b[1;32m    762\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0misstring\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    763\u001b[0m         \u001b[0mpattern\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mp\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 764\u001b[0;31m         \u001b[0mp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msre_parse\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mparse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mflags\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    765\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    766\u001b[0m         \u001b[0mpattern\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/anaconda3/lib/python3.8/sre_parse.py\u001b[0m in \u001b[0;36mparse\u001b[0;34m(str, flags, state)\u001b[0m\n\u001b[1;32m    938\u001b[0m     \u001b[0;31m# parse 're' pattern into list of (opcode, argument) tuples\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    939\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 940\u001b[0;31m     \u001b[0msource\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTokenizer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    941\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    942\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mstate\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/anaconda3/lib/python3.8/sre_parse.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, string)\u001b[0m\n\u001b[1;32m    230\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    231\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 232\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__next\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    233\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__next\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    234\u001b[0m         \u001b[0mindex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/anaconda3/lib/python3.8/sre_parse.py\u001b[0m in \u001b[0;36m__next\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    243\u001b[0m                 \u001b[0mchar\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecoded_string\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    244\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mIndexError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 245\u001b[0;31m                 raise error(\"bad escape (end of pattern)\",\n\u001b[0m\u001b[1;32m    246\u001b[0m                             self.string, len(self.string) - 1) from None\n\u001b[1;32m    247\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mindex\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31merror\u001b[0m: bad escape (end of pattern) at position 0"
     ]
    }
   ],
   "source": [
    "re.search('\\\\', s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcf2e59a",
   "metadata": {},
   "source": [
    "* The problem here is that the backslash escaping happens twice, first by the Python interpreter on the string literal and then again by the regex parser on the regex it receives.\n",
    "\n",
    "    * The Python interpreter is the first to process the string literal '\\\\'. It interprets that as an escaped backslash and passes only a single backslash to re.search().\n",
    "    * The regex parser receives just a single backslash, which isn’t a meaningful regex, so the messy error ensues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "215249e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='\\\\'>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('\\\\\\\\', s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "87b0cdf7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 4), match='\\\\'>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(r'\\\\', s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "223a2174",
   "metadata": {},
   "source": [
    "__It’s good practice to use a raw string to specify a regex in Python whenever it contains backslashes.__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "100e5cec",
   "metadata": {},
   "source": [
    "## Anchors\n",
    "Anchors are zero-width matches. They don’t match any actual characters in the search string, and they don’t consume any of the search string during parsing. Instead, an anchor dictates a particular location in the search string where a match must occur."
   ]
  },
  {
   "cell_type": "raw",
   "id": "0d74e504",
   "metadata": {},
   "source": [
    "^           \n",
    "\\A Anchor a match to the start of <string>."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "554dd5e5",
   "metadata": {},
   "source": [
    "* When the regex parser encounters ^ or \\A, the parser’s current position must be at the beginning of the search string for it to find a match.\n",
    "\n",
    "* In other words, regex ^foo stipulates that 'foo' must be present not just any old place in the search string, but at the beginning:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "bbae21a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='foo'>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import re\n",
    "re.search('^foo', 'foobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "ea6913a9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.search('^foo', 'barfoo'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "fd3e1761",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='foo'>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('\\Afoo', 'foobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "76424bea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.search('\\Afoo', 'barfoo'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d4bfff8",
   "metadata": {},
   "source": [
    "^ and \\A behave slightly differently from each other in MULTILINE mode. You’ll learn more about MULTILINE mode below in the section on flags."
   ]
  },
  {
   "cell_type": "raw",
   "id": "9953e28c",
   "metadata": {},
   "source": [
    "$\n",
    "\\Z  Anchor a match to the end of <string>."
   ]
  },
  {
   "cell_type": "raw",
   "id": "44ce5afa",
   "metadata": {},
   "source": [
    "When the regex parser encounters $ or \\Z, the parser’s current position must be at the end of the search string for it to find a match. \n",
    "\n",
    "Whatever precedes $ or \\Z must constitute the end of the search string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef289a58",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('bar$', 'foobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99968dd6",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('bar$', 'barfoo'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50717370",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('bar\\Z', 'foobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58de7c95",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('bar\\Z', 'barfoo'))"
   ]
  },
  {
   "cell_type": "raw",
   "id": "125158a8",
   "metadata": {},
   "source": [
    "As a special case, $ (but not \\Z) also matches just before a single newline at the end of the search string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "eca2055a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='bar'>"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('bar$', 'foobar\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a87c4f3f",
   "metadata": {},
   "source": [
    "In this example, 'bar' isn’t technically at the end of the search string because it’s followed by one additional newline character. But the regex parser lets it slide and calls it a match anyway. This exception doesn’t apply to \\Z."
   ]
  },
  {
   "cell_type": "raw",
   "id": "44062d82",
   "metadata": {},
   "source": [
    "\\b: Anchors a match to a word boundary."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6fff2a2",
   "metadata": {},
   "source": [
    "* __\\b__ asserts that the regex parser’s current position must be at the beginning or end of a word. A word consists of a sequence of alphanumeric characters or underscores ([a-zA-Z0-9_]), the same as for the \\w character class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "18bd7117",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search(r'\\bbar', 'foo bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f8bf04a",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search(r'\\bbar', 'foo.bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "4d2e0fc4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.search(r'\\bbar', 'foobar'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "54687021",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='bar'>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(r'bar\\b', 'foobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "e642fcc9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='foo'>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(r'foo\\b', 'foo bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "0771084d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='foo'>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(r'foo\\b', 'foo.bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "0b48ab70",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.search(r'foo\\b', 'foobar'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "740fc068",
   "metadata": {},
   "source": [
    "* Using the \\b anchor on both ends of the 'regex' will cause it to match when it’s present in the search string as a whole word:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc368485",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search(r'\\bbar\\b', 'foo bar baz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "01aa6bdf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(4, 7), match='bar'>"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(r'\\bbar\\b', 'foo(bar)baz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "b18d9e43",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.search(r'\\bbar\\b', 'foobarbaz'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cfd1ff94",
   "metadata": {},
   "outputs": [],
   "source": [
    "\\B: Anchors a match to a location that isn’t a word boundary."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f2d4ee4",
   "metadata": {},
   "source": [
    "* __\\B__ does the opposite of \\b. It asserts that the regex parser’s current position must not be at the start or end of a word:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38d58e80",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search(r'\\Bfoo\\B', 'foo'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5f91585",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search(r'\\Bfoo\\B', '.foo.'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c9888f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search(r'\\Bfoo\\B', 'barfoobaz')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "100e8796",
   "metadata": {},
   "source": [
    "## Quantifiers\n",
    "\n",
    "A __quantifier__ metacharacter immediately follows a portion of a 'regex' and indicates how many times that portion must occur for the match to succeed."
   ]
  },
  {
   "cell_type": "raw",
   "id": "37523d88",
   "metadata": {},
   "source": [
    "* Matches zero or more repetitions of the preceding regex. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09e21f80",
   "metadata": {},
   "source": [
    "* For example, a* matches zero or more 'a' characters. That means it would match an empty string, 'a', 'aa', 'aaa', and so on.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3b0dc0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('foo-*bar', 'foobar')                     # Zero dashes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36f3b780",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('foo-*bar', 'foo-bar')                    # One dash"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8004912a",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('foo-*bar', 'foo--bar')                   # Two dashes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60dde34e",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('foo.*bar', '# foo $qux@grault % bar #')    #.* matches everything between 'foo' and 'bar'"
   ]
  },
  {
   "cell_type": "raw",
   "id": "36ccc300",
   "metadata": {},
   "source": [
    "+ Matches one or more repetitions of the preceding regex."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4770e437",
   "metadata": {},
   "source": [
    "* This is similar to *, but the quantified regex must occur at least once:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5d45444",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('foo-+bar', 'foobar'))              # Zero dashes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71422cd2",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('foo-+bar', 'foo-bar')                    # One dash"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a767006",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('foo-+bar', 'foo--bar')                   # Two dashes"
   ]
  },
  {
   "cell_type": "raw",
   "id": "232f8ef0",
   "metadata": {},
   "source": [
    "？Matches zero or one repetitions of the preceding regex."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "041a73a7",
   "metadata": {},
   "source": [
    "* Again, this is similar to * and +, but in this case there’s only a match if the preceding regex occurs once or not at all:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4068e0bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('foo-?bar', 'foobar')                     # Zero dashes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "239d28b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('foo-?bar', 'foo-bar')                    # One dash"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40c82588",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('foo-?bar', 'foo--bar'))            # Two dashes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dc6d2c4",
   "metadata": {},
   "source": [
    "* Here are some more examples showing the use of all three quantifier metacharacters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c3cc2c3",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.match('foo[1-9]*bar', 'foobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0a201e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.match('foo[1-9]*bar', 'foo42bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2418ff49",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.match('foo[1-9]+bar', 'foobar'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49493c99",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.match('foo[1-9]+bar', 'foo42bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90e57f76",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.match('foo[1-9]?bar', 'foobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d08e2d6d",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.match('foo[1-9]?bar', 'foo42bar'))"
   ]
  },
  {
   "cell_type": "raw",
   "id": "afe9b431",
   "metadata": {},
   "source": [
    "*?\n",
    "+?\n",
    "??\n",
    "The non-greedy (or lazy) versions of the *, +, and ? quantifiers."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0f96b5d",
   "metadata": {},
   "source": [
    "* When used alone, the quantifier metacharacters *, +, and ? are all greedy, meaning they produce the longest possible match. Consider this example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b465415",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('<.*>', '%<foo> <bar> <baz>%')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bba42a8",
   "metadata": {},
   "source": [
    "* If you want the shortest possible match instead, then use the non-greedy metacharacter sequence *?:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd2807de",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('<.*?>', '%<foo> <bar> <baz>%')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2664ac9",
   "metadata": {},
   "source": [
    "There are lazy versions of the + and ? quantifiers as well:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32b95d96",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('<.+>', '%<foo> <bar> <baz>%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66e49bc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('<.+?>', '%<foo> <bar> <baz>%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1af44ac6",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('ba?', 'baaaa')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "df6c52fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('ba??', 'baaaa')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "aaed4bc7",
   "metadata": {},
   "source": [
    "{m} Matches exactly m repetitions of the preceding regex."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "009814ca",
   "metadata": {},
   "source": [
    "* This is similar to * or +, but it specifies exactly how many times the preceding regex must occur for a match to succeed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7cd2315",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('x-{3}x', 'x--x'))                # Two dashes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c34d6f5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('x-{3}x', 'x---x')                      # Three dashes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d13ebbc5",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(re.search('x-{3}x', 'x----x'))              # Four dashes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff6e0175",
   "metadata": {},
   "source": [
    "{m,n} Matches any number of repetitions of the preceding regex from m to n, inclusive."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62986d33",
   "metadata": {},
   "source": [
    "* In the following example, the quantified regex is -{2,4}. The match succeeds when there are two, three, or four dashes between the 'x' characters but fails otherwise:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "e83bad7f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1  x-x        None\n",
      "2  x--x       <re.Match object; span=(0, 4), match='x--x'>\n",
      "3  x---x      <re.Match object; span=(0, 5), match='x---x'>\n",
      "4  x----x     <re.Match object; span=(0, 6), match='x----x'>\n",
      "5  x-----x    None\n"
     ]
    }
   ],
   "source": [
    "for i in range(1, 6):\n",
    "...     s = f\"x{'-' * i}x\"\n",
    "...     print(f'{i}  {s:10}', re.search('x-{2,4}x', s))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1c549d5",
   "metadata": {},
   "source": [
    "![打开方式](m_n.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6f88248",
   "metadata": {},
   "source": [
    "* If you omit all of m, n, and the comma, then the curly braces no longer function as metacharacters. {} matches just the literal string '{}':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82c7fabb",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('x{}y', 'x{}y')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a45127a",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('x{foo}y', 'x{foo}y')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d644988",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('x{a:b}y', 'x{a:b}y')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "060f2531",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('x{1,3,5}y', 'x{1,3,5}y')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "03d8aef2",
   "metadata": {},
   "source": [
    "{m,n}?\n",
    "The non-greedy (lazy) version of {m,n}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b53a600f",
   "metadata": {},
   "source": [
    "* {m,n} will match as many characters as possible, and {m,n}? will match as few as possible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd152ac6",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('a{3,5}', 'aaaaaaaa')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a144c337",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='aaa'>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import re\n",
    "re.search('a{3,5}?', 'aaaaaaaa')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dce5fd9",
   "metadata": {},
   "source": [
    "## Grouping Constructs and Backreferences\n",
    "Grouping constructs break up a regex in Python into subexpressions or groups. This serves two purposes:\n",
    "\n",
    "* Grouping: A group represents a single syntactic entity. Additional metacharacters apply to the entire group as a unit.\n",
    "\n",
    "* Capturing: Some grouping constructs also capture the portion of the search string that matches the subexpression in the group. You can retrieve captured matches later through several different mechanisms."
   ]
  },
  {
   "cell_type": "raw",
   "id": "2c49a94c",
   "metadata": {},
   "source": [
    "(<regex>)\n",
    "Defines a subexpression or group."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00c1cada",
   "metadata": {},
   "source": [
    "* This is the most basic grouping construct. A regex in parentheses just matches the contents of the parentheses:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "744e957d",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(bar)', 'foo bar baz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8612a342",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('bar', 'foo bar baz')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c7712f2",
   "metadata": {},
   "source": [
    "### Treating a Group as a Unit\n",
    "A quantifier metacharacter that follows a group operates on the entire subexpression specified in the group as a single unit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43ae63cd",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(bar)+', 'foo bar baz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3872bd5a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(4, 9), match='barrr'>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('bar+', 'foo barrr baz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a72d2b87",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(bar)+', 'foo barbar baz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80de904c",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(bar)+', 'foo barbarbarbar baz')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1dbe7a40",
   "metadata": {},
   "source": [
    "### Question Onsite: difference between bar+ and (bar)+ "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4a1ac7a",
   "metadata": {},
   "source": [
    "Now take a look at a more complicated example. The regex (ba[rz]){2,4}(qux)? matches 2 to 4 occurrences of either 'bar' or 'baz', optionally followed by 'qux'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "692e600a",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(ba[rz]){2,4}(qux)?', 'bazbarbazqux')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fca2ad0",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(ba[rz]){2,4}(qux)?', 'barbar')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bca485b",
   "metadata": {},
   "source": [
    "### Question Onsite: (foo(bar)?)+(\\d\\d\\d)? "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed6810c0",
   "metadata": {},
   "source": [
    "![打开方式](q2.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "500f92e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(foo(bar)?)+(\\d\\d\\d)?', 'foofoobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f2f7182",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(foo(bar)?)+(\\d\\d\\d)?', 'foofoobar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f270e2d",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search('(foo(bar)?)+(\\d\\d\\d)?', 'foofoo123')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2ea1747",
   "metadata": {},
   "source": [
    "### Capturing Groups\n",
    "Grouping isn’t the only useful purpose that grouping constructs serve. Most (but not quite all) grouping constructs also capture the part of the search string that matches the group. You can retrieve the captured portion or refer to it later in several different ways.\n",
    "\n",
    "Remember the match object that re.search() returns? There are two methods defined for a match object that provide access to captured groups: .groups() and .group()."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34ed9891",
   "metadata": {},
   "source": [
    "__m.groups()__: Returns a tuple containing all the captured groups from a regex match"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c47f026c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 12), match='foo,quux,baz'>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m = re.search('(\\w+),(\\w+),(\\w+)', 'foo,quux,baz')\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16d46495",
   "metadata": {},
   "source": [
    "* Because the (\\w+) expressions use grouping parentheses, the corresponding matching tokens are captured. \n",
    "* To access the captured matches, you can use .groups(), which returns a tuple containing all the captured matches in order:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "4ef3097c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'foo,quux,baz'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "667ae56b",
   "metadata": {},
   "source": [
    "__m.group(n):__ Returns a string containing the <n>th captured match."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5866dbf5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'foo'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8210a8df",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'quux'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8a4d080",
   "metadata": {},
   "source": [
    "### Question: what about m.group(0) ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "b19649e2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'foo,quux,baz'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b437071",
   "metadata": {},
   "source": [
    "__m.group(n1, n2, ...)__: Returns a tuple containing the specified captured matches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08e7fc60",
   "metadata": {},
   "outputs": [],
   "source": [
    "m.groups()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "4c7271a2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('quux', 'baz')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(2, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "200997f6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('baz', 'quux', 'foo')"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(3, 2, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7d5a5be",
   "metadata": {},
   "outputs": [],
   "source": [
    "(m.group(3), m.group(2), m.group(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b08869e",
   "metadata": {},
   "source": [
    "__Backreferences:__ You can match a previously captured group later within the same regex using a special metacharacter sequence called a backreference."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2074344",
   "metadata": {},
   "source": [
    "__\\n:__ Matches the contents of a previously captured group.\n",
    "\n",
    "* Within a regex in Python, the sequence \\n, where n is an integer from 1 to 99, matches the contents of the n th captured group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "fa1c690f",
   "metadata": {},
   "outputs": [],
   "source": [
    "regex = r'(\\w+),\\1'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "2b17dcce",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 7), match='foo,foo'>\n"
     ]
    }
   ],
   "source": [
    "m = re.search(regex, 'foo,foo')\n",
    "print(m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "db7cf936",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'foo'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "d9b0b6a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 7), match='qux,qux'>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m = re.search(regex, 'qux,qux')\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "4407c609",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'qux'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "1b1f102d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "m = re.search(regex, 'foo,qux')\n",
    "print(m)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f9bec2c",
   "metadata": {},
   "source": [
    "__Note:__ Any time you use a regex in Python with a numbered backreference, it’s a good idea to specify it as a raw string. Otherwise, the interpreter may confuse the backreference with an octal value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "5cdb8ee6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(0, 3), match='d#d'>\n"
     ]
    }
   ],
   "source": [
    "print(re.search(r'([a-z])#\\1', 'd#d'))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d12121bb",
   "metadata": {},
   "source": [
    "* the match fails because Python misinterprets the backreference \\1 as the character whose octal value is one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "b07f9873",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0o1'"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "oct(ord('\\1'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fac83c73",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search(r'([a-z])#\\1', 'd#d')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2d9323a",
   "metadata": {},
   "source": [
    "* Numbered backreferences are one-based like the arguments to .group(). Only the first ninety-nine captured groups are accessible by backreference. The interpreter will regard \\100 as the '@' character, whose octal value is 100."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a29ed83f",
   "metadata": {},
   "source": [
    "### Other Grouping Constructs\n",
    "\n",
    "The (regex) metacharacter sequence shown above is the most straightforward way to perform grouping within a regex in Python. The next section introduces you to some enhanced grouping constructs that allow you to tweak when and how grouping occurs."
   ]
  },
  {
   "cell_type": "raw",
   "id": "8d8c2d26",
   "metadata": {},
   "source": [
    "(?P<name><regex>)： Creates a named captured group."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae9176d8",
   "metadata": {},
   "source": [
    "* This metacharacter sequence is similar to grouping parentheses in that it creates a group matching ‘regex’ that is accessible through the match object or a subsequent backreference.\n",
    "\n",
    "* The difference in this case is that you reference the matched group by its given symbolic name instead of by its number.\n",
    "\n",
    "Earlier we saw the example with three captured group numbers 1,2,3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "c2d06cc6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('foo', 'quux', 'baz')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m = re.search('(\\w+),(\\w+),(\\w+)', 'foo,quux,baz')\n",
    "m.groups()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "3273886f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('foo', 'quux', 'baz')"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(1, 2, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c476ec80",
   "metadata": {},
   "source": [
    "The following effectively does the same thing except that the groups have the symbolic names w1, w2, and w3:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "c88fba64",
   "metadata": {},
   "outputs": [],
   "source": [
    "m = re.search('(?P<w1>\\w+),(?P<w2>\\w+),(?P<w3>\\w+)', 'foo,quux,baz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "24e9b998",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('foo', 'quux', 'baz')"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.groups()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "fa89ebb0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('foo', 'quux', 'baz')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.groups('w1')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "e2f8f1c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('foo', 'quux', 'baz')"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group('w1', 'w2', 'w3')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccdc857d",
   "metadata": {},
   "source": [
    "* Any name specified with this construct must conform to the rules for a __Python identifier__, and each name can only appear __once per regex.__"
   ]
  },
  {
   "cell_type": "raw",
   "id": "ad104332",
   "metadata": {},
   "source": [
    "(?P=<name>): Matches the contents of a previously captured named group"
   ]
  },
  {
   "cell_type": "raw",
   "id": "bcb2a70c",
   "metadata": {},
   "source": [
    "The (?P=<name>) metacharacter sequence is a backreference, similar to \\<n>, except that it refers to a named group rather than a numbered group."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b40ccba7",
   "metadata": {},
   "source": [
    "Here again is the example from above, which uses a numbered backreference to match a word, followed by a comma, followed by the same word again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "9df6a566",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 7), match='foo,foo'>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m = re.search(r'(\\w+),\\1', 'foo,foo')\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "963f572d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'foo'"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.group(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "026a8054",
   "metadata": {},
   "source": [
    "The following code does the same thing using a named group and a backreference instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "44b020c8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 7), match='foo,foo'>"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m = re.search(r'(?P<word>\\w+),(?P=word)', 'foo,foo')\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b217d3e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "m.group('word')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5067b8e9",
   "metadata": {},
   "source": [
    "__Note: The angle brackets (< and >) are required around name when creating a named group but not when referring to it later, either by backreference or by .group()__"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d007cde",
   "metadata": {},
   "source": [
    "## Modified Regular Expression Matching With Flags"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a333837",
   "metadata": {},
   "source": [
    "* Most of the functions in the re module take an optional 'flags' argument. This includes the function you’re now very familiar with, re.search()."
   ]
  },
  {
   "cell_type": "raw",
   "id": "9a66f342",
   "metadata": {},
   "source": [
    "re.search(<regex>, <string>, <flags>): \n",
    "Scans a string for a regex match, applying the specified modifier <flags>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "656e83c6",
   "metadata": {},
   "source": [
    "* Flags modify regex parsing behavior, allowing you to refine your pattern matching even further."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d32bceb5",
   "metadata": {},
   "source": [
    "Supported Regular Expression Flags\n",
    "* The table below briefly summarizes the available flags. All flags except re.DEBUG have a short, single-letter name and also a longer, full-word name:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8740653",
   "metadata": {},
   "source": [
    "![打开方式](flag.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66809cda",
   "metadata": {},
   "source": [
    "re.I\n",
    "re.IGNORECASE: Makes matching case insensitive."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24015213",
   "metadata": {},
   "source": [
    "* When IGNORECASE is in effect, character matching is case insensitive:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "dc37ae98",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='aaa'>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('a+', 'aaaAAA')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "b474c9ac",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(3, 6), match='AAA'>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('A+', 'aaaAAA')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "2b6e32fe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 6), match='aaaAAA'>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('a+', 'aaaAAA', re.I)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "60a74ea8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 6), match='aaaAAA'>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('A+', 'aaaAAA', re.IGNORECASE)"
   ]
  },
  {
   "cell_type": "raw",
   "id": "92c9b689",
   "metadata": {},
   "source": [
    "re.M\n",
    "re.MULTILINE：Causes start-of-string and end-of-string anchors to match at embedded newlines. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2623ab9",
   "metadata": {},
   "source": [
    "By default, the ^ (start-of-string) and $ (end-of-string) anchors match only at the beginning and end of the search string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "2ee9774f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='foo'>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = 'foo\\nbar\\nbaz'\n",
    "re.search('^foo', s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "66bfb5cc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "foo\n",
      "bar\n",
      "baz\n"
     ]
    }
   ],
   "source": [
    "print(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "70e609cd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.search('^bar', s))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "4170d13b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.search('foo$', s))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "605e32bc",
   "metadata": {},
   "source": [
    "* In this case, even though the search string 'foo\\nbar\\nbaz' contains embedded newline characters, only 'foo' matches when anchored at the beginning of the string, and only 'baz' matches when anchored at the end.\n",
    "\n",
    "* If a string has embedded newlines, however, you can think of it as consisting of multiple internal lines. In that case, if the MULTILINE flag is set, the ^ and \\$ anchor metacharacters match internal lines as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "20bbdbd9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(4, 7), match='bar'>"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('^bar', s, re.MULTILINE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "76c00d68",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='foo'>"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('foo$', s, re.M)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e1ce763",
   "metadata": {},
   "source": [
    "__Note: The MULTILINE flag only modifies the ^ and \\$ anchors in this way. It doesn’t have any effect on the \\A and \\Z anchors__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "4ebd62df",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<re.Match object; span=(4, 7), match='bar'>\n"
     ]
    }
   ],
   "source": [
    "s = 'foo\\nbar\\nbaz'\n",
    "print(re.search('^bar', s, re.MULTILINE))"
   ]
  },
  {
   "cell_type": "raw",
   "id": "60772d47",
   "metadata": {},
   "source": [
    "re.S\n",
    "re.DOTALL: Causes the dot (.) metacharacter to match a newline."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5e9cbd6",
   "metadata": {},
   "source": [
    "* Remember that by default, the dot metacharacter matches any character except the newline character. The DOTALL flag lifts this restriction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "b482c751",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.search('foo.bar', 'foo\\nbar'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "246758d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 7), match='foo\\nbar'>"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('foo.bar', 'foo\\nbar', re.DOTALL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "bb4bd661",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 7), match='foo\\nbar'>"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('foo.bar', 'foo\\nbar', re.S)"
   ]
  },
  {
   "cell_type": "raw",
   "id": "ec47f265",
   "metadata": {},
   "source": [
    "re.X\n",
    "re.VERBOSE：Allows inclusion of whitespace and comments within a regex."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66424dda",
   "metadata": {},
   "source": [
    "The VERBOSE flag specifies a few special behaviors:\n",
    "\n",
    "The regex parser ignores all whitespace unless it’s within a character class or escaped with a backslash.\n",
    "\n",
    "If the regex contains a # character that isn’t contained within a character class or escaped with a backslash, then the parser ignores it and all characters to the right of it.\n",
    "\n",
    "What’s the use of this? It allows you to format a regex in Python so that it’s more readable and self-documenting."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9988b51b",
   "metadata": {},
   "source": [
    "Suppose you want to parse phone numbers that have the following format:\n",
    "\n",
    "* Optional three-digit area code, in parentheses\n",
    "* Optional whitespace\n",
    "* Three-digit prefix\n",
    "* Separator (either '-' or '.')\n",
    "* Four-digit line number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "1712c53c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 8), match='414.9229'>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regex = r'^(\\(\\d{3}\\))?\\s*\\d{3}[-.]\\d{4}$'\n",
    "re.search(regex, '414.9229')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88a27233",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.search(regex, '414-9229')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "b438290f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 13), match='(712)414-9229'>"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(regex, '(712)414-9229')"
   ]
  },
  {
   "cell_type": "raw",
   "id": "9f044e0f",
   "metadata": {},
   "source": [
    "But r'^(\\(\\d{3}\\))?\\s*\\d{3}[-.]\\d{4}$' is an eyeful, isn’t it? \n",
    "Using the VERBOSE flag, you can write the same regex in Python like this instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "04ff1178",
   "metadata": {},
   "outputs": [],
   "source": [
    "regex = r'''^               # Start of string\n",
    "...             (\\(\\d{3}\\))?    # Optional area code\n",
    "...             \\s*             # Optional whitespace\n",
    "...             \\d{3}           # Three-digit prefix\n",
    "...             [-.]            # Separator character\n",
    "...             \\d{4}           # Four-digit line number\n",
    "...             $               # Anchor at end of string\n",
    "...             '''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "c011ecee",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 8), match='414.9229'>"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(regex, '414.9229', re.VERBOSE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "ec2e2bcd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 13), match='(712)414-9229'>"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(regex, '(712)414-9229', re.X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b1b1576",
   "metadata": {},
   "outputs": [],
   "source": [
    "re.DEBUG: Displays debugging information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "447f624e",
   "metadata": {},
   "source": [
    "* The DEBUG flag causes the regex parser in Python to display debugging information about the parsing process to the console:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "4b16aac8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LITERAL 102\n",
      "LITERAL 111\n",
      "LITERAL 111\n",
      "ANY None\n",
      "LITERAL 98\n",
      "LITERAL 97\n",
      "LITERAL 114\n",
      "\n",
      " 0. INFO 12 0b1 7 7 (to 13)\n",
      "      prefix_skip 3\n",
      "      prefix [0x66, 0x6f, 0x6f] ('foo')\n",
      "      overlap [0, 0, 0]\n",
      "13: LITERAL 0x66 ('f')\n",
      "15. LITERAL 0x6f ('o')\n",
      "17. LITERAL 0x6f ('o')\n",
      "19. ANY\n",
      "20. LITERAL 0x62 ('b')\n",
      "22. LITERAL 0x61 ('a')\n",
      "24. LITERAL 0x72 ('r')\n",
      "26. SUCCESS\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 7), match='fooxbar'>"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('foo.bar', 'fooxbar', re.DEBUG)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4eda90b0",
   "metadata": {},
   "source": [
    "When the parser displays LITERAL nnn in the debugging output, it’s showing the ASCII code of a literal character in the regex. In this case, the literal characters are 'f', 'o', 'o' and 'b', 'a', 'r'."
   ]
  },
  {
   "cell_type": "raw",
   "id": "26fa0a82",
   "metadata": {},
   "source": [
    "re.A\n",
    "re.ASCII\n",
    "re.U\n",
    "re.UNICODE\n",
    "re.L\n",
    "re.LOCALE\n",
    "\n",
    "Specify the character encoding used for parsing of special regex character classes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8ee2e23",
   "metadata": {},
   "source": [
    "* Several of the regex metacharacter sequences (\\w, \\W, \\b, \\B, \\d, \\D, \\s, and \\S) require you to assign characters to certain classes like word, digit, or whitespace. \n",
    "\n",
    "* The flags in this group determine the encoding scheme used to assign characters to these classes. The possible encodings are ASCII, Unicode, or according to the current locale."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "066c436e",
   "metadata": {},
   "source": [
    "* Why is character encoding so important in the context of regexes in Python? Here’s a quick example.\n",
    "    * You learned earlier that \\d specifies a single digit character. The description of the \\d metacharacter sequence states that it’s equivalent to the character class [0-9]. \n",
    "    * That happens to be true for English and Western European languages, but for most of the world’s languages, the characters '0' through '9' don’t represent all or even any of the digits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "c1e83b84",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'१४६'"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = '\\u0967\\u096a\\u096c'\n",
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fa6dc38",
   "metadata": {},
   "source": [
    "* For the regex parser to properly account for the Devanagari script, the digit metacharacter sequence \\d must match each of these characters as well.\n",
    "* The Unicode Consortium created Unicode to handle this problem. Unicode is a character-encoding standard designed to represent all the world’s writing systems. \n",
    "* All strings in Python 3, including regexes, are Unicode by default."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a879c7d4",
   "metadata": {},
   "source": [
    "So then, back to the flags listed above. These flags help to determine whether a character falls into a given class by specifying whether the encoding used is ASCII, Unicode, or the current locale:\n",
    "\n",
    "* __re.U and re.UNICODE__ specify Unicode encoding. Unicode is the default, so these flags are superfluous. They’re mainly supported for backward compatibility.\n",
    "\n",
    "* __re.A and re.ASCII__ force a determination based on ASCII encoding. If you happen to be operating in English, then this is happening anyway, so the flag won’t affect whether or not a match is found.\n",
    "\n",
    "* __re.L and re.LOCALE__ make the determination based on the current locale. Locale is an outdated concept and isn’t considered reliable. Except in rare circumstances, you’re not likely to need it.\n",
    "\n",
    "Using the default Unicode encoding, the regex parser should be able to handle any language you throw at it. In the following example, it correctly recognizes each of the characters in the string '१४६' as a digit:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbb392fa",
   "metadata": {},
   "source": [
    "Here’s another example that illustrates how character encoding can affect a regex match in Python. Consider this string:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "0f316a23",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'schön'"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = 'sch\\u00f6n'\n",
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76f5e7f6",
   "metadata": {},
   "source": [
    "* 'schön' (the German word for pretty or nice) contains the 'ö' character, which has the 16-bit hexadecimal Unicode value 00f6. This character isn’t representable in traditional 7-bit ASCII.\n",
    "\n",
    "* If you’re working in German, then you should reasonably expect the regex parser to consider all of the characters in 'schön' to be word characters. But take a look at what happens if you search s for word characters using the \\w character class and force an ASCII encoding:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "5cd09b68",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 3), match='sch'>"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('\\w+', s, re.ASCII)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "b5dde577",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 5), match='schön'>"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('\\w+', s, re.UNICODE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "00a46259",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<re.Match object; span=(0, 5), match='schön'>"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search('\\w+', s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bebd11b",
   "metadata": {},
   "source": [
    "![打开方式](special_character.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1bd5af1c",
   "metadata": {},
   "source": [
    "![打开方式](sets.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c25c378e",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
